The ability to form images at different depths within a display, whether real or perceived, has been the subject of significant and ongoing research and development in the quest to provide display technology capable of replicating or augmenting the depth effects conferred by normal human sight.
Three-dimensional or multi-focal plane displays are known to provide numerous advantages or capabilities unavailable with conventional two-dimensional displays. The manner in which human beings process visual information has been the subject of extensive and prolonged research in an attempt to understand this complex process. This research has included the effects depth or ‘apparent depth’ provided by three dimensional or multi-focal pane displays.
The term preattentive processing has been coined to denote the act of the subconscious mind in analysing and processing visual information which has not become the focus of the viewer's conscious awareness.
When viewing a large number of visual elements, certain variations or properties in the visual characteristics of elements can lead to rapid detection by preattentive processing. This is significantly faster than requiring a user to individually scan each element, scrutinising for the presence of the said properties. Exactly what properties lend themselves to preattentive processing has in itself been the subject of substantial research. Colour, shape, three-dimensional visual clues, orientation, movement and depth have all been investigated to discern the germane visual features that trigger effective preattentive processing.
Researchers such as Triesman [1985] conducted experiments using target and boundary detection in an attempt to classify preattentive features. Preattentive target detection was tested by determining whether a target element was present or absent within a field of background distractor elements. Boundary detection involves attempting to detect the boundary formed by a group of target elements with a unique visual feature set within distractors. It maybe readily visualised for example that a red circle would be immediately discernible set amongst a number of blue circles. Equally, a circle would be readily detectable if set amongst a number of square shaped distractors. In order to test for preattentiveness, the number of distractors as seen is varied and if the search time required to identify the targets remains constant, irrespective of the number of distractors, the search is said to be preattentive. Similar search time limitations are used to classify boundary detection searches as preattentive.
A widespread threshold time used to classify preattentiveness is 200-250 msec as this only allows the user opportunity for a single ‘look’ at a scene. This timeframe is insufficient for a human to consciously decide to look at a different portion of the scene. Search tasks such as those stated above maybe accomplished in less than 200 msec, thus suggesting that the information in the display is being processed in parallel unattendedly or pre-attentively.
However, if the target is composed of a conjunction of unique features, i.e. a conjoin search, then research shows that these may not be detected preattentively. Using the above examples, if a target is comprised for example, of a red circle set within distractors including blue circles and red squares, it is not possible to detect the red circle preattentively as all the distractors include one of the two unique features of the target.
Whilst the above example is based on a relatively simple visual scene, Enns and Rensink [1990] identified that targets given the appearance of being three dimensional objects can also be detected preattentively. Thus, for example a target represented by a perspective view of a cube shaded to indicate illumination from above would be preattentively detectable amongst a plurality of distractor cubes shaded to imply illumination from a different direction. This illustrates an important principle in that the relatively complex, high-level concept of perceived three dimensionality may be processed preattentively by the sub-conscious mind.
In comparison, if the constituent elements of the above described cubes are re-orientated to remove the apparent three dimensionality, subjects cannot preattentively detect targets which have been inverted for example. Additional experimentation by Brown et al [1992] confirm that it is the three dimensional orientation characteristic which is preattentively detected. Nakaymyama and Silverman [1986] showed that motion and depth were preattentive characteristics and that furthermore, stereoscopic depth could be used to overcome the effects of conjoin. This reinforced the work done by Enns Rensink in suggesting that high-level information is conceptually being processed by the low-level visual system of the user. To test the effects of depth, subjects were tasked with detecting targets of different binocular disparity relative to the distractors. Results showed a constant response time irrespective of the increase in distractor numbers.
These experiments were followed by conjoin tasks whereby blue distractors were placed on a front plane whilst red distractors were located on a rear plane and the target was either red on the front plane or blue on the rear plane for stereo colour (SC) conjoin tests, whilst stereo and motion (SM) trials utilised distractors on the front plane moving up or on the back plane moving down with a target on either the front plane moving down or on the back plane moving up.
Results showed the response time for SC and SM trials were constant and below the 250 msec threshold regardless of the number of distractors. The trials involved conjoin as the target did not possess a feature unique to all the distractors. However, it appeared the observers were able to search each plane preattentively in turn without interference from distractors in another plane.
This research was further reinforced by Melton and Scharff [1998] in a series of experiments in which a search task consisting of locating an intermediate-sized target amongst large and small distractors tested the serial nature of the search whereby the target was embedded in the same plane as the distractors and the preattentive nature of the search whereby the target was placed in a separate depth plane to the distractors.
The relative influence of the total number of distractors present (regardless of their depth) verses the number of distractors present solely in the depth plane of the target was also investigated. The results showed a number of interesting features including the significant modification of the response time resulting from the target presence or absence. In the target absence trials, the reaction times of all the subjects displayed a direct correspondence to the number of distractors whilst the target present trials did not display any such dependency. Furthermore, it was found that the reaction times in instances where distractors were spread across multiple depths were faster than for distractors located in a single depth plane.
Consequently, the use of a plurality of depth/focal planes as a means of displaying information can enhance preattentive processing with enhanced reaction/assimilation times.
Known three-dimensional displays seek to provide binocular depth cues to the viewer via a variety of techniques including separate head-mounted displays located directly in front of each eye, lenticular displays and holography. Unfortunately, each of these possesses certain limitations. Head-mounted displays add ergonomic inconvenience, reduce the viewer's peripheral awareness and are often cumbersome and can cause nausea, headaches and/or disorientation. Lenticular displays are only really effective at oblique viewing angles and holography is currently limited to displaying static images.
A further implementation of a three-dimensional display is referred to herein as a ‘combination display’ is configured with two displays of known type located at differing distances from a half-silvered mirror. The orientation of the displays is such that one display is visible along a ray axis passing directly through the half-silvered mirror, whilst the other display is visible along a ray axis reflected from the mirror's surface.
A composite image may be formed therefore from the respective images shown on both displays. The differing distances of the displays from the half-silvered mirror leads to the perception that the images are located at different depths within the composite image scene viewed. Such systems are unavoidably bulky and cumbersome in comparison to conventional single screen displays in order to house the two separate displays without any physical overlap.
Furthermore, the luminance of the image transmitted to the viewer is attenuated by the intrinsic transmissive qualities of the half silvered mirror, requiring the use of a more intense back-light (or similar illumination means) in each display.
Additional difficulties arise from the generation of a parallax error proportional to the distance between the image planes, which is exacerbated by increasing the display separation to increase the ‘depth’ of the scene perceived by the viewer. Conversely, if the displays are brought into close proximity, moiré interference effects mar the resultant image.
Displays such as those described above create a three dimensional effect by displaying images on a number of optically overlapping, essentially planar image or boundary planes. Whilst this offers an enhancement to the depth cues afforded by a conventional display, it would be desirable to display an image at any desired depth within the display, rather than being restricted to the physical display image planes.
This problem has been partially addressed by applying a technique commonly referred to as ‘depth fusion’ to the above described ‘combination display’ i.e., a configuration of two separate displays and half-silvered mirror. Depth fusion involves displaying two identical images on separate overlapping image planes such that the alignment and magnification of the two-layer image are perceived as coterminous when viewed along the viewer's fixation axis, i.e., a line from the viewed image extending equidistantly to the fovea of each retina to the mid-point between the viewer's eyes. In a combination display, the overlapping coterminous images are discerned through the half-silvered mirror.
Contrary to an intuitive analysis, it has been found that varying the relative luminance distributions between the two overlapped images causes the perceived location of the resultant image to be at a point between the two image planes.
Whilst this clearly provides a beneficial effect, the above-described shortcomings of combination displays using a half-silvered mirror, i.e., parallax distortion, excessive bulk and luminance attenuation are still present.
Consequently, there is a need to provide the ability to display images at a variable depth without the physical constraints imposed by the above described prior art.
All references, including any patents or patent applications, cited in this specification are hereby incorporated by reference. No admission is made that any reference constitutes prior art. The discussion of the reference states what their authors assert, and the applicants reserve the right to challenge the accuracy and pertinency of the cited documents. It will be clearly understood that, although a number of prior art publications are referred to herein, this reference does not constitute an admission that any of these documents forms parts of the common general knowledge in the art in any country.
It is an object of the present invention to address the foregoing problems.
Further aspects and advantages of the present invention will become apparent from the ensuing description which is given by way of example only.